NTCCRT: A concurrent constraint framework for soft-real time music interaction

Writing music interaction systems is not easy because their concurrent processes usually access shared resources in a non-deterministic order, often leading to unpredictable behavior. Using Pure Data (Pure Data) and Max/MSP, it is possible to program concurrency; however, it is difficult to synchronize processes based on multiple criteria. Process calculi such as the Non-deterministic Timed Concurrent Constraint (ntcc) calculus, overcome that problem by representing, declaratively, the synchronization of multiple criteria as constraints. In this article, we propose the framework Ntccrt, as a new alternative to manage concurrency in Pure Data and Max/MSP. Ntccrt is a real-time capable interpreter for ntcc. Using Ntccrt binary plugins in Pure Data, we executed models for machine improvisation and signal processing. We also analyzed two case studies: one of a machine improvisation system and one of a signal processing system. We found out that performance of both case studies is compatible with soft real-time music interaction; it means, a musician can interact with Ntccrt without noticeable delays during the interaction

On the Information Geometry of Audio Streams with Applications to Similarity Computing

International audienceThis paper proposes methods for information processing of audio streams using methods of information geometry. We lay the theoretical groundwork for a framework allowing the treatment of signal information as information entities, suitable for similarity and symbolic computing on audio signals. The theoretical basis of this paper is based on the information geometry of statistical structures representing audio spectrum features, and specifically through the bijection between the generic families of Bregman divergences and that of exponential distributions. The proposed framework, called Music Information Geometry allows online segmentation of audio streams to metric balls where each ball represents a quasi-stationary continuous chunk of audio, and discusses methods to qualify and quantify information between entities for similarity computing. We define an information geometry that approximates a similarity metric space, redefine general notions in music information retrieval such as similarity between entities, and address methods for dealing with non-stationarity of audio signals. We demonstrate the framework on two sample applications for online audio structure discovery and audio matching

Using Multidimensional Sequences For Improvisation In The OMax Paradigm

International audienceAutomatic music improvisation systems based on the OMax paradigm use training over a one-dimensional sequence to generate original improvisations. Different systems use different heuristics to guide the improvisation but none of these benefits from training over a multidimensional sequence. We propose a system creating improvisation in a closer way to a human improviser where the intuition of a context is enriched with knowledge. This system combines a probabilistic model taking into account the multidimen-sional aspect of music trained on a corpus, with a factor oracle. The probabilistic model is constructed by interpolating sub-models and represents the knowledge of the system, while the factor oracle (structure used in OMax) represents the context. The results show the potential of such a system to perform better navigation in the factor oracle, guided by the knowledge on several dimensions

Guided Improvisation as Dynamic Calls to an Offline Model

(Abstract to follow

Cross-Modal Variational Inference For Bijective Signal-Symbol Translation

International audienceExtraction of symbolic information from signals is an active field of research enabling numerous applications especially in the Musical Information Retrieval domain. This complex task, that is also related to other topics such as pitch extraction or instrument recognition, is a demanding subject that gave birth to numerous approaches , mostly based on advanced signal processing-based algorithms. However, these techniques are often non-generic, allowing the extraction of definite physical properties of the signal (pitch, octave), but not allowing arbitrary vocabularies or more general annotations. On top of that, these techniques are one-sided, meaning that they can extract symbolic data from an audio signal, but cannot perform the reverse process and make symbol-to-signal generation. In this paper, we propose an bijective approach for signal/symbol translation by turning this problem into a density estimation task over signal and symbolic domains, considered both as related random variables. We estimate this joint distribution with two different variational auto-encoders, one for each domain, whose inner representations are forced to match with an additive constraint, allowing both models to learn and generate separately while allowing signal-to-symbol and symbol-to-signal inference. In this article, we test our models on pitch, octave and dynamics symbols, which comprise a fundamental step towards music transcription and label-constrained audio generation. In addition to its versatility, this system is rather light during training and generation while allowing several interesting creative uses that we outline at the end of the article

Modèles Probabilistes pour l'Interaction entre agents

Dans un contexte d'interaction Humain-Machine, notre objectif est l'élaboration d'un modèle probabiliste d'interaction générique et vraisemblable capable de commander à la fois un Agent Conversationnel Animé (ACA) dans un cadre d'interaction et un Agent Musical Créatif (AMC) dans un contexte d'improvisation musicale